DOCUMENT RESUME 



ED 366 989 



CS 214 241 



AUTHOR 
TITLE 



Burkhal ter , Nancy 

Assessing Editorials: An Instrument To Aid in the 

Task* 

Nov 93 

20p.; Paper presented at the Annual Meeting of the 
Speech Communication Association (79th, Miami Beach, 
FL, November 18-21, 1993). 

Speeches/Conference Papers (150) — Guides - 
Classroom Use - Teaching Guides (For Teacher) (052) 
— Tests/Evaluation Instruments (160) 



PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



MFO1/PC01 Plus Postage. 

^Editorials; Grading; Higher Education; Holistic 
Evaluation; Journal ism Education; ^Student 
Evaluation; ^Writing Evaluation 
Writing Contexts 



ABSTRACT 



Two kinds of instruments, atomistic versus holistic, 



offer different approaches to the grading of writing. This paper 
briefly discusses the pros and cons of these two philosophies of 
grading. In addition, the paper presents a holistic, primary-trait 
scoring instrument that has been used successfully to judge 
editorials by breaking down the analysis into three essential 
components: claims, data, and warrants. The paper concludes that 
using this primary- trai t instrument aids journalism teachers not only 
in pinpointing weaknesses in editorials but also in designing 
remedial solutions for students. Criteria for judging quality of 
claims, data, and warrants; two student-written editorials, are 
attached. (Contains 15 references.) (Author/RS) 



ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft yc ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft * ft ft ft ft ftftftft ft ft 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ftftftxft ft ft ft ft ft ft ft ft ft ft ftftft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft 



ASSESSING EDITORIALS: AN INSTRUMENT TO AID IN THE TASK 



By 



Nancy Burkhalter, PhD 



University of Wyoming 
Communication and Mass Media Department 
P.O. Box 3904 University Station 
Laramie, WY 82071-3904 
(307) 766-3122 



Presented at the Speech Communication Association Convention 

Miami , Florida 
November 18-20, 1993 



U S DCPAHTMENT OF EOUCATlON 



OH<© Of EdoCitK>«il Research irxJ ImpfOvemcni 




EDUCATIONAL RESOURCES INFORMATION 



CENTER (ERIC) 



Tr« s document hjj been reproduced as 
deceived Ifom the person c o*ganual<on 
CKtgm»|irvg it 



( Mm >r changes have bean made to »mpro*e 

■ep'.XluClrOn Quality 



• Pomll Ot v»ew O' OPmrOns Stated this dOCu 

mem 3o not neces&aniy »eD»©ae"i ott»c«ai 
OER' pos«t«on o» policy 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



ABSTRACT 

Two kinds of instruments, atomistic versus holistic, offer 
vastly different approaches to the grading of writing. This 
paper briefly discusses the pros and cons of these two 
philosophies of grading. In addition, it presents a holistic, 
primary-trait scoring instrument that has been used successfully 
to judge editorials by breaking down the analysis into three 
essential components, claims, data, and warrants. The paper 
concludes that using this primary-trait instrument aids 
journalism teachers not only in pinpointing weaknesses in 
editorials but also in designing remedial solutions for students. 
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ASSESSING EDITORIALS : AN INSTRUMENT TO AID IN THE TASK 

Assessing writing has been a problem for many journalism 
teachers. While we as experienced readers and graders may know 
intuitively that students have made progress since writing their 
last story, we can't always say explicitly how or why. This can 
leave students frustrated and unable to mend their ways in the 
next draft. 

Two kinds of instruments, atomistic versus holistic, offer 
vastly different approaches to the grading of compositions. This 
paper presents the pros and cons of these two philosophies of 
grading. In addition, it presents a holistic, primary-trait 
scoring instrument that has been used successfully to judge 
editorials by breaking down the analysis into three essential 
components: claims, data, and warrants. 
How best to measure growth in writing? 

When faced with an essay to grade and students who need 
guidance before attempting a subsequent draft, we have two 
methods of grading that can potentially be informative about the 
progress of the student: a holistic vs. an atomistic approach. 

Adherents to the holistic philosophy deem it superior 
because it is potentially more valid and informative (Lloyd- 
Jones, 1977). Supporters of the atomistic approach, however, 
find holistic approach lacking because of its seemingly amorphous 
methodology. They claim that breaking down writing into its 
rhetorical constituents is more reliable and renders good, hard 
data that are easier to communicate to students and account for 
grades . 
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Atomistic instruments 

A good example of an atomistic approach to grading 
compositions is measuring mean sentence length. Designed in the 
60s, this was thought to be the best indicator of linguistic 
maturity. To verify this assumption, Hunt (1965) analyzed 
writing samples ranging from 4th graders to adults and found that 
average sentence length increased with age, but he was not 
convinced it was a sensitive enough indicator of syntactic 
maturity. He subsequently devised another system of measurement, 
the 'minimal terminable unit,' or T-unit. A T-unit is a main 
clause, with or without a subordinate clause attached to it. 
Dividing the number of words by the number of T-units reveals the 
level of syntactic maturity. 

Hunt's system was cosidered to be misleading since it 
ignored the relationship between sentence length, topic, purpose, 
genre, and psychological distance. Rosen (1969), for instance, 
found that "the nature of the writing task influences mean T-unit 
length at least as much as the writer's maturity does" (cited in 
Watson, 1983, p. 132) . 

Although the notion of T-units has its detractors, it has 
been used to investigate syntactic maturity across discourse 
types, and found that persuasive writing was syntactically denser 
than other types of writing (Cooper and Watson (1980); Perron 
(1977); and San Jose, 1973). Perhaps this density can be 
accounted for by Crowhurst and Piche's (1979) study, in which 6th 
and 10th grade students wrote persuasive, descriptive, and 
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narrative discourse for different audiences (teacher vs. best 
friend) . They found the persuasive writing to be syntactically 
more complex, and that the more distant the audience, the longer 
the T-units. Although these data help us gain some understanding 
about how writing tasks differ, this atomistic approach does 
little to help us determine where students have gone wrong. 
Holistic instruments 

Despite the staying power of Hunts T-units, other 
researchers prefer an approach that does not focus purely on 
clause length, transitions, or any other "countable" aspect of 
writing. They deem this method misleading. When writing is 
analyzed in terms of its constituents, say, as Hunt's T-unit 
measures syntactic structures, one necessarily studies a subskill 
of the process, and conclusions drawn from such a methodology in 
writing can misdirect and misinform the researcher. Therefore, 
opponents to atomistic instruments conclude, any piece of writing 
needs to be studied and analyzed in its entirety. 

Holistic instruments offer an alternate approach to 
atomistic instruments. Generally, a holistic instrument is any 
that still requires assessment of linguistic, rhetorical, or 
information features of a piece of writing but never calls upon 
the reader to stop reading to count or tally any one feature. 
The reader is given only a list of specified features or a 
general guideline of what to focus on while reading. The reader 
then rates the various features according to the level of 
proficiency for that particular trait (Cooper, 1977). 
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Various forms of holistic scales have been developed. 
Cooper (1977) cites several that vary in the kinds and number of 
features to be investigated in a piece. One of those scales, the 
primary trait scoring instrument, isolates subcategories of the 
universe of discourse (e.g., cohesion), and rates writing samples 
in terms of their aptness within the prescribed range. Although 
this approach breaks up the scoring process into categories just 
as its atomistic counterpart does, this one focuses on aspects of 
a composition that can be judged more holistically . For 
instance, if I want to judge a composition's cohesion, I must 
read the entire composition and make a holistic judgment based on 
my impressions of that component. In contrast, an atomistic 
instrument may have the reader count the number of connectors and 
transitions throughout the essay. But according to proponents of 
the holistic approach, cohesion can be signaled in less overt, 
less quantifiable ways and thus may be overlooked or under- 
estimated by an atomistic instrument. 
Assessing editorials 

While all good writing shares many of the same features, 
e.g., cohesiveness, certain genres have special essential 
components. A good editorial, for instance, is made up of a 
strong thesis statement, topic sentences that support that r ^aim, 
and facts, anecdotes, and illustrations that enhance the strength 
of that claim. The stronger the "pieces" of the essay, the more 
persuasive the editorial. 

A primary-trait scoring instrument works quite well when 
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assessing an editorial since these global elements, while in some 
ways are discrete, are very much intertwined and interdependent. 
In comparison, the atomistic features of syntax and mechanics 
(punctuation and spelling) have no direct relation to the 
persuasiveness of an essay and certainly are at too specific a 
level to even consider relevant in measuring, say, the strength 
of an argument. 
The instrument 

This instrument, adapted from Connor's (1990) and McCann's 
(1989) instruments, was designed to assess claims, data, and 
warrants (Toulmin, 1958), traits found in any kind of persuasive 
writing (See Appendix 1 for instrument used) . 

Claims (a.k.a. proposition) are "conclusions whose merits we 
are seeking to establish" (Toulmin, 1958, p. 97) and as 
"assertions put forward publicly for general acceptance" (Toulmin 
et al., 1984, p. 29). A claim can be identified by asking these 
questions: What are you claiming, where do you stand, what 
position are you asking us to agree to? (Toulmin, Rieke, Janik, 
1979) . 

Data (a.k.a. evidence and grounds) are interpreted as 
support for the claim in the form of facts, experience, 
statistics, or occurrences. Data answer the question, what 
information are you going on to make such a claim? 

Warrants are construed as amplification or further 
explanation of the data that clearly connects the data to the 
claim. These arise from laws of nature, legal principles, rules 
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of thumb, and engineering formulas and so on. Although often 
implied, warrants can often be discovered by asking how the 
writer can justify moving from these grounds to that claim. 
Given facts such as these, any person would conclude such and 
such. 

Analysis focused on these elements since a good editorial 
contains all three. Yet they are not equally weighted as to 
their importance in the success the essay. 

Claims, for instance, are more akin to the thesis statement 
of any good composition. They make assertions or declarations 
and are always capable of being proven true or untrue. If well 
stated, it orients the reader to the subject and viewpoint of the 
writer. If poorly stated, it leaves the reader somewhat at sea 
but usually able to rely on context to garner that information. 
Data are somewhat more important since they delineate the major 
points the writer wants to advance in favor of the argument. 
Usually, the fewer points mentioned, the less convincing the 
arguments; however, more does not necessarily guarantee success 
either. They must be relevant and strong. 

Warrants are by far the most difficult aspect of persuasive 
writing yet the most crucial in convincing one's audience. They 
are the most difficult because they require a writer be aware of 
any possible objections to the viewpoint stated in the claim, and 
be exhaustive in the kinds of facts, illustrations, and anecdotes 
proffered. Citing a statistic then warrants the evidence to the 
claim. While supplying strong, relevant, and warranted data is 
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an element of any good writing no matter what the genre, it is 
especially important when a desired outcome depends on the 
strength of the arguments, it becomes even more important. 

The scale can be broken up into more or fewer levels, 
depending on the assignment, the students, their level of 
proficiency, and time factors. 
Applying the instrument 

Appendix 2 presents two editorials on the same subject, 
although from opposite viewpoints, written for a feature-writing 
course. 

After making an outline Steve's editorial, I gave each 
element in the instrument the following rating: 
Claim - Cloning is dangerous because it can get out of hand. 
(Claim = 6) 

Data - We screwed up the water hyacinth. 

We screwed up the nutria. (Data = 2 because they are not 
exactly analogous to the human cloning situation. What are 
statistics on the cloning of cows, for instance?) 
Warrant - Since we screwed up before with something so 
dangerous, we can screw up again. (Warrant = 2) 

Because the student's claim, that cloning human genes can 
get out of hand, is clearly and unambiguously stated, thus 
earning him a 6. Yet his two pieces of data rated only a 2 
(possibly a 4) by were not as convincing as they could have been. 
While it is true that our mishandling of the water hyacinth and 
the nutria should be warnings and thus support his claim, it 
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would be more powerful to point out if we had already tried 
experiments on humans or something closer to us on the 
evolutionary scale, such as cattle. His only warrant, that we 
"screwed up 11 before so we could easily "screw up again" is enough 
to earn him a 2 in that category. Yet we can easily see he is 
not providing enough data or enough warrants to fully convince 
the reader of his claim. Although it is a good attempt at a 
first draft, the essay needs fleshing out at both the data and 
warrant levels. 

The second essay by Cynde is in favor of human cloning. 
Clajm - Cloning embryos is good because it could lead to medical 
advances and relieve suffering. (Claim = 6) 

Pfrta - (1) Cloning can increase changes of success of in vitro- 
fertilization. 

(Possible warrant - Since this technique is so expensive and 
has such a low success rate, it could lower the costs and raise 
reliability of the procedure). 

(2) Storage of embryo can help women in case of 
chemotherapy . 

(Possible warrant - This would mean chemotherapy would not 
end her chances of fertility*) 

(3) Can be used in screening in embryos for birth 
defects. (Data =4) ^ 

(Possible warrant - Destroying a clone is less traumatic and 
ethically more acceptable than the destroying the original 
embryo . 

ERIC * 1 



Warrants - (Warrants =0). 

Because Cynde's editorial also clearly stated her position, 
that cloning could contribute to many medical advances, her claim 
was rated a 6. She earned a 4 for data because, although she 
provided three good arguments to support her claim, she could 
have provided more information, for example, about just exactly 
what in-vitro fertilization is. She received a 0 for warrants, 
because , as her outline reveals, she had no warrants to really 
lock the reader into her viewpoint. My outline includes possible 
warrants to show how much more effective the essay could be were 
they included. (Previous research (author, 1992) has shown that 
students have a much more difficult time with warrants than with 
any other aspect of persuasive writing. See author (1993) for a 
discussion of the reasons for this.) 
CONCLUSION 

This instrument pinpoints the strengths and weaknesses at 
all the various levels of editorials. Getting a breakdown of 
this information by means of this primary-trait scoring 
instrument helps students see why and where their compositions 
are not as convincing as they could be and helps teachers 
identify where they should concentrate their remedial efforts. 
Giving a holistic grade, even with copious comments, may not 
inform the student of strengths and weaknesses as clearly as this 
instrument does. 
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APPENDIX 1. CRITERIA FOR JUDGING QUALITY OF CLAIMS , DATA/ AND 
WARRANTS 

CLAIM - conclusions whose merits we are seeking to establish and 
assertions put forward for general acceptance 

0 No clear position exists of the writer's assertion, 
preference, or view and context does not help clarify it. 

2 Writer's position is poorly formulated but reader is 
reasonably sure what the paper is about because of context. 

4 A topic sentence exists and reader is reasonably sure what 
the paper is about on the strength of the topic sentence alone, 
regardless of context. 

6 A very clear, concise position is given as a topic sentence. 
The reader is very certain what paper is about. 

DATA - support in the form of experience, facts, statistics, or 
occurrences . 

0 No data are offered that are relevant to the claim. 

2 Scant data (one or two pieces) are offered but what exist are 
usually relevant to claim. Irrelevant data are excluded. 

4 Numerous pieces of data (minimum three) in support of the 
claim are offered. They are relevant but not necessarily 
convincing or complete. The writer leaves much for the reader to 
infer from the data. The writer may offer data which are not 
complete enough to allow the reader to determine their 
significance. 

6 Numerous pieces of data (minimum three) are complete and 
accurate as well as explicitly and convincingly connected to the 
claim. 

WARRANT - amplification or further explanation of data 
0 No warrant is offered. 

2 An attempt is made to elaborate at least one element of the 
data but the reader is left to infer more. 

4 More than one piece of data is explained but the explanation 
is weak or lacks thoroughness. 

6 The writer explains data in such a way that it is clear how 
they support the data. At least one piece of data is convincing 
and complete. 
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APPENDIX 2 

Steve's Essay 

It may finally come to pass as so many other "science 
fiction" ideas have. Man has learned to fly, go to outer space, 
create hybrid flowers and organisms, and now we may have the 
biggest ingredient of science fiction come true: HUMAN CLONING. 

We have to be very careful with human cloning, it could get 
out of hand, like many other experiments gone awry. Once it has 
been discovered and employed, man has the ultimate power. 

What if this power would fall into the wrong hands? 

What if Hitler had this power? 

What if the fat computer hacker in Jurassic Park actually 
got away with the stolen vials of genes? 

We need to further delve into these "what ifs" before we 
take the final step in man creating man. 

Don't worry, the day is not upon us yet, but it is closing 
in. Dr. Robert J. Stillman at the George Washington University 
Medical Center in Washington, D.C., has recently been able to 
split 17 abnormal human embryos into 48 single cells. A few of 
these cells actually reached the 32 cell stage in an artificial 
gel coating, but the cells only lived for six days before dying. 
The deaths could be attributed to the cells being abnormal to 
begin with. 

Researchers are able to genetically clone organisms from the 
micro level on up, but humans are far more complicated than that. 
It could take years and years, but human cloning is on the way. 
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What if they screw up? 

What could Stephen King to with this? 

On a much simpler level than RNA, DNA and gene splitting we 
have screwed up just with intentional species introductions. The 
water hyacinth was introduced into Florida as a decorative water 
plant, and later introduced into the wild, which it took over. 
Now ponds and lakes in the region are literally being choked by 
the renegade plant. 

Next we imported what turned out to be the scourge of 
Florida, the nutria. This furry, beaver-looking animal was 
introduced to the region to get rid of the water hyacinth because 
it feeds on the plant. 

The little rodent also took over the region and has become a 
bigger problem than rats ever were in Florida. 

Even when we experiment with importing god-made creatures, 
the results can be far from what was expected. When we 
experiment with human-altered beings we may be getting in over 
our heads. 

Sure we can grow bigger, tastier, redder apples by 
genetically altering the plants, but an apple is a far cry from a 
human being. 

Man is always going forward. That is how America and the 
speed of sound were discovered. Progress is fine, but we must 
know when to draw the line. 

That line is now. 
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Cynde's Essay 

Last week the New York Times exposed a seemingly modest 
fertility breakthrough, launching the biggest ethical debate in 
medicine since the first test tube baby 15 years ago. 

The first cloning of a human embryo was performed by two 
George Washington University scientists at the American Fertility 
Society in Montreal three weeks ago. 

Since the Times broke the news, we've heard the cries from 
ethicists who have conjured up such Brave New World/Jurassic Park 
nightmares as baby farms and embryo factories which mass produce 
human spare parts. 

But under tight regulation by the creation of a presidential 
commission, cloning of human embryos could lead to important 
medical advances and relieve human suffering within the next 
couple of years. 

Ironically, cloning is a relatively simple procedure that 
has been performed for years with plants, pigs, cows and rabbits. 
In fact, cloning cattle is a booming business in the United 
States . 

Cloning occurs naturally when a woman conceives identical 
twins. To clone humans, scientists divide cells in two and coat 
with an artificial additive. The cells then continue to 
multiply. 

The most immediate way cloning can be helpful is with in- 
vitro fertilization. This is the primary reason cloning was 
designed. A woman with one embryo has a 10-20 percent chance of 
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getting pregnant through in-vitro fertilization. But by cloning 
the embryo three or four times, her chance of being impregnated 
increases tremendously. 

Cloning can be applied to other ethical medical practices as 
well. A woman who is about to undergo chemotherapy or another 
procedure which would render her sterile could have an embryo 
cloned and stored for future use. 

Cloning can also be used when screening embryos at risk for 
inherited diseases, such as cystic fibrosis or hemophilia. The 
current screening technique involves a procedure which can be 
terminal to the embryo. But by having a clone around to test for 
the disorder, there is no harm done. 

There is a real fear that cloning will go too far. Women 
who wish to have a child though artificial insemination have 
catalogs available that list the qualities of the donors. There 
is even a catalog made up solely of Nobel prize winners! But 
with cloning, many suspect that someday greedy entrepreneurs will 
sell the embryos of these Nobel prize winners. 

That is why government regulation is important. More than 
25 countries have commissions that set guidelines on the 
advancement of reproductive technology. Under President Carter, 
a similar commission was developed to make policy on the most 
controversial of medical issues, such as euthanasia. 

We are in the dawn of the eugenics era. Under the super- 
vision of the commission formed by President Clinton, infertile 
couples will be able to increase their chances of conceiving a 
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child and we will have the peace of mind knowing that this new 
technology will not be abused. 
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